A Practical Approach to Uncertainty Handling and Estimate Acquisition in Model-based Prediction of System Quality

Our earlier research indicated the feasibility of applying the PREDIQT method for model-based prediction of impacts of architectural design changes on system quality. The PREDIQT method develops and makes use of so called prediction models, a central part of which are the “Dependency Views” (DVs) – weighted trees representing the relationships between architectural design and the quality characteristics of a target system. The values assigned to the DV parameters originate from domain expert judgements and measurements on the system. However fine grained, the DVs contain a certain degree of uncertainty due to lack and inaccuracy of empirical input. This paper proposes an approach to the representation, propagation and analysis of uncertainties in DVs. Such an approach is essential to facilitate model fitting (that is, adjustment of models during verification), identify the kinds of architectural design changes which can be handled by the prediction models, and indicate the value of added information. Based on a set of criteria, we argue analytically and empirically, that our uncertainty handling approach is comprehensible, sound, practically useful and better than any other approach we are aware of. Moreover, based on experiences from PREDIQT-based analyses through industrial case studies on real-life systems, we also provide guidelines for use of the approach in practice. The guidelines address the ways of obtaining empirical estimates as well as the means and measures for reducing uncertainty of the estimates. Keywords-uncertainty, system quality prediction, modeling, architectural design, change impact analysis, simulation.